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ABSTRACT ' 

In order to understand better the , elemental .processes 
involved in beginning reading (specif ically^-how cognitive skills are 
arCquired and whetherr a relationship exists-among basic skills), 
Robert Calfee and colleagues conducted a first-grade reading diary 
study. Forty target children in ^973.-1 974 and 50 children in 
1974-1975 were observed as they were being taught to read and were 
given systematic and continuaJ. tests designed according to the 
development of^ their reading skills. This pap^r describes the 
construction of the network of tests whicb were used for the study 
and which were designed to identify the specific skills directly 
related to reading acquisition and to control for those general 
abilities which influence a student •s performance on virtually any 
task.. Although data from the study have not been completely analyzed, 
it is apparent both that the elemental processes being measured did 
provide the basis of beginning reading achievement and that- this 
method of assessment offers teachers precise diagnostic information 
about the instructional needs of beginning students. (JH) 
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A case history study has the potential of providing a wealth of long- 
^ itudinal iriforination not .available in other designs. The First Grade Reading 

ON Diary Study is a unique exatnple of a case history approach aimed toward pro- 

^ viding description of how children acquire begihning reading skills. This 

study was carried oat under the auspices of the Carnegie Corporation by ^ 
Dr. Robert Calfee and his colleagues at Stanford University. It aimed 
toward a better understanding of the elemental processes in beginning read- 

ing. How are these cognitive skills acquired? IS there a relationship 

I • • 

among basic reading skills?- . ■ 

-■ ■ • * • . ' 

Forty target children in 1973-74 and 50 children in 1974-75 from first 
grade classrooms in the San Jose Unified School District weir.e observed while 
they were being taught to read, and were given systematic and continual tests, 
designed according to the development of their reading skills. The class- 
rooms were selected from low td moderate income neighborhood schools using 
teachers with a variety of classroom philosophies, teaching styles, aid 
curriculum choices. Variations in target classro^s, ^nd in teaching styles 

and method, were included to isolate the effects of instruction used to 

\ 

promote ac^quisition of decoding skills. Students were selected from among 
those who we predicted would , learti to read, but were not already doing so - 
ipon entering first grade. The children represented a range of ethnic 
backgrounds. > * — . • • 

Two primary data collecting systems were used: (1) a comprehensive 
1^ observational system designed to capturfe the historical events withiji the . 

claesroom and (2)' a network of tests constructed to yield precise measures 
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of specific, componential skills. acquired along the path toward reading. 
Observational systems generally employed have tended to be narrowlj., con- 
fined to limited or sporadic classroom activities, to be insensitive to be- 
havioral frequencies, to be tied to a'priori categories, or to be built upon 
.observer-defined scales of subjective indices. For our case history approach', 
a comprehensive system of objective observation, RAMOS (Reading and iMathe- 
matics Obsepation System [Calfee & Hoover, 19741), «as utili^ed to provide " 
a computer-compatible sum..ary of classroom activity and structure. Obser- " 
vations took place weekly 'during reading instruction and focused on many 
■features in each of the classrooms. The flexibility of RAMOS eateries 
permitted analyses of the activities of the target children, the function- 
ing of the teacher, the usage .of space, the extent of and time allotted to 
instruction, the teacher's feedback procedures, and a number of other coded 
behaviors and situations. These features of RAm'oS enabled us to accumulate 
an objective, historical description of what was^oing on in each first 
grade classroom throughout the school year. 

Concurrent with the accumulation of descriptive observations, the 
Reading Diary Study determined precisely what each student learned from > 
week to week, and to what extent this knowledge had generaliEed. This 
challenge required tests that would pinpoint separable components along the 
continuum from noqreading to reading. 

• Tests currently ,us^d for assessment- of reading achievement possess 
great predictive reliability, but are not sensitive to the elemental pro- 
cesses underlying^tery of larger skills. Most existing tests are con-, 
structed so that in order to do well a student must bring to bear a complex 
variety of skills and knowledge. For our purposes, we needed specific 
kinds of information quite independent of norm-referenced performance that • 



3 

Piontkowski. - Clinical Assessment^> . . ' 6 18 

mask precise.. individual differences in basic competencies. Our goal was to 
identify the specific skills directly related to reading acquisition and to 
control for those general abilities that influence a student's performance 
on virtually any tasks, confounding his competence on elemental tasks. 
General abilities' include knowledge of how a task is to be performed, a 
set toward ..sjieed rather than accuracy, and availability of coding systems 
for processing incoming information and reducing short-term memo"ry load. 

We developed an assessment system to obtain what Glaser (1970) has 
called "the outcomes of learning"--the specific performance-skills of the 
target studlrtts on pitacisely defined measures. The underlying rationale 
is in line withi Gagne's (1970) description of distortion-free measurement 
in whifih p^rfcyrmancer is not influenced by any variable other than the one 
explicitly defined. Such tests are "clean" (Calfee, 1976)--the task re- 
quiremenqs are simplified, the relevant cu^s are salient, and the substance 
of the task i^ designed to match the student's past learning experiences as 
nearly as possible/ 

Before examining the hierarchical structure across subtests, we will 
focus on the explicit controls for contamination within each test-cell or 
subtest. Irl order %o direct the chil-d's attention to the relevant cues in 
each 3ubte5>t, two sample items were provided with corrective feedback 
immediately prior to the subtest. These sample items helped clarify direc- 
tions by example. To control for a child^'s set toward ^eed rather than > 
accuracy, all testing was done individually with the pace determined by the 
tester. More than one^ correct item was frequently included among each 
multiple-cnoi,ce array in* order to stress inclusive search'-strategies . The 
boredom and fatigue brought pn by lengthy, repdititive subtests were avoided 
by succinct test-cells ' of only four items that quickly indicated mastery or 
ignorance of ^/specific skill. 
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At the same time, w6 developed a system to avoid presenting subtests 
beyond the competence of the individual student, oT^taking the^more skilled- 
student through a lengthy series of subtests, that measured skills he had 
already mastered, A dj/namic or clinical fij^mat was created which allowed 
the tester to plot each child's course through the subtests on which he 
was successful while minimizing his exposure to subtests that were too easy 
or too difficult for him. Wittrock (1970) has referred to this as a^ 
"multistage decision process." The tester follows a prescribed branching 
rule making decisipns sequentially which affect subsequent decisions. The 
length and/ sequence* of testing varies according to each child's performance, 
and continues until the branching rule cannot be satisfied, that is, until 
the chili has mastered the most difficult subtests, and his level of corn- 
pet cntnerifhas^b Jen completely defined by the intersection of subtests mastered 
and subtests failed. In this way, untested skills are assumed to have been 
mastered or not yeyt learned according to their position in the hierarchy 
and th^ student's responses. 



The initial phase of test construction was based on a theoretical sketch 
c#^' the logical^ classification system in which the concepts or beginning read- 
ing are generally taught. Thus, the sequence and Structure of the assessment 
scheme reflect the range of skills directly related to instruction. Analyses 
of methods and materials allowed for predicting the steps a child takes as 
he progresses from nonreader to beginning reader. Within this series, two 
sub^oals are apparent: (1) the child Ijeams an alphabetic foundation of 
the written system and letter-sound correspondences in order to analyze 
and synthesize the letter combinations that lead to reading, and (2) the 
child learns to read sequential strings of words to finally arrive at mean- 
ing and comprehension. ^ Within each of these subgoals exist the elemental 
processes and* skills that must be mastered in learning to read. 
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The first step toward definition of the elemental p|tocesses was an 
analysis of the essential attributes of the content of beginning reading iead 
ing to development of a gena^al classification scheme for the asff^sraent net- 
work. The primary substantivp variable appeared to be orthographic 'composi- 
tion. We hypothesized that word oeadability is a serial progression dii- 
ficulty starting with consonpt-VOwel-consonant word^and followed by/ con- 
sonant blends and digraphs, vowel contrasts, and polysy^abic words. This 
separation of words iy spelling pattern defined the primary categoric/ bound- 
ary, and specified which woyd-items could be included in each test-sCheme 
within the network while explicitly e^xcluding all other word-items. Figure 1 
provides a diagram of the priir^ry dimension of the testing network, 

A multi-dimensional design of subtests was generated according to the 
inherent dimensions of vari^ability within the content and form of each word- 
category. That is, each category was analyzed into dimensions according to 
the structural boundaries imposed by the Orthography of the words, and b^ 
the tasks which represent the specific skills requisite to mastery of that 
category. In general, we incWed and extended the guidelines proposed by 
Baker (1974). One dimension consists of all the^ligible content specifi- 
cations: all areas of separable content required to assure njaster of .that 
category of letter-sound correspondence. Another dimension is concerned 
with the form in which the learner must display his knoivledge, all ways the 
child might demonstrate those behaviors indicating mastery^ A third dimen- 
sion measures the child's ability to generalize his specific knowledge to 
materials differing in degree of temporal availability and familiarity 
Figure 2 pr&^ents the three dimensional madel of the consonant-vowel- 
consonant test scheme. - ' 
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Figure 3 
about hefej 



Each of these content and* tes^ppnse dimensions werfe then factored, into 
the significant elementis of that climension..- Underlying'^the ability to read 
the wjbole \iord, the child needs to learn liow 'to deal with specific components 
oH t|lat w6rd--the initial consoiTkr\t, final consonant, medial vowel. Within 

// ■ - • , •• ^ ^ : . 

theldirtiension called ,"letter-sound environment" we generated measures of 

• , , * . .' 

eadh of these elemental processes. We factoretd "task" variables in order 
-< . * ' ' ■ ■ ^ 

assess differentially the child's ability to recognize the correct 

response from, 4in array of letters or words, to produte verbdHy the correct ; . 

'^espoase, and. to write the appropriate let^j pr'word. In order to neasure 

^transfer of knowledge about consonaht-vowel-conspnai^t wor4s , we factored 

"materials" into three elements: pictured and verbalized stimuli, verbal- 

ized labels, aiid nonsense words. The three-dimensiona^l framework of the 

consonajnt-vowel-consbnant category is presented in a nfested scheme in Figure 3. 

Each subtest is defined by the crossing of three speci^ed elements within 

the ^ree dimensions of form and content. Humphreys (1962) and Guttman J^1969) 

^ have defined this method of test construction as subtest definition by ex-^ 

plicit specification of the combinations of elements. Subtes^ts are gjener^ted ' 

^ ^ciroG^ the dimensions by specific limi'tatioas on jthe items for each subtest ■ 

accprding to the restrictions imposed* by ^1:he structural definitions. New 

cells are created by holding two dimensional 'elements constant and varying ^ 

oiily the third. Each item within each subtest must fit the precise factorial 

* definition. By moving a child through t)hie sulatests , systematic sources of 

^difficulty Are .added and the •child ' s performance indicates the' precise situ- 

ational alteration that separates his mastery from ignorance. In this way, 

**.."■' • • ° 

each testing session determines the stvdent 's- knowledge of beginning reading 

defined by the fac toxical. f^ihierisions of that- test. By starting ^witlTsVbtests 
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Figure 4 
about here 
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yieldi^ng flow error frequency and systematically introducing sources of dlf fi- ^ 
culty unidimensionally^" the child's; performance across subtests provides a 
' 'diagnostic write-up of his elemental skills. 

Thus, the test oLructure per.se is an integral part of the entire 
instrument; the structure of the test defines precisely what it is- the child 
can and cannot do. The multidimensional network |f, subtests" mirror the 
•elemental subskills' that are required for mastery of the gre^^er skill. 
Each subtest measures a factorially unique process so tha4: wWen the nested 
subskills are combined again, they b(^come t:hat skill'. When 'talcen separately, 
each subtest j^vides diagnosis withi,n definition of the' dimensional elements; 



when grouped, the' subtests provide a-measure of the child's knowledge across 
elements and dimensi^orts. By examining the child's performance,- the tester 
is able td arrive^at an Unbiased assessment of the child's specific abilities 
on an explicitly defined hierarchy of ^reading skills. 

In* addition to the multi-dimensional test-schemes for word-categories, 
entry and probing sysj^ms surround each scheme. Figure 4 provides a dia- 
grammatic plan of the more extensive network. Entry tests were constructed 
according to the word-item limitations imposed by the word-category ^ and in 
addition, contained irregular functional words to assess the child's ability 
to recognize and read certain high .frequency words not included in any of 

the categorical schemes. The entry tepts did not measure analytic abilities, 

\ ' , 

but were limited to two performance taiks; identification of a verbalized 
I . . , ^ ■ • ' ' , - \ ' ' • 

strimuli from a printed arxay, and abili\ty to read a list of words. The 

\ ' 

entry tests were vehicles for making branching decisions about, which test^ 
scheme to present to the child, and at , what Inavel of difficulty to enter. 
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A comprehensive system of probing tests was developed surrounding the 
consonant-vowel-consonant test scheme in order to hone in on , specific sources^ 
of difficu'lty. . These included tests of letter-sound corresp'ondences, letter 
naming, letter writing, and letter discrimination. 

As indicated across the top of Figure 4, we also tested the target 
students' ability to read prose derived from words of similar orthographic 
patterns to "fun" picture books^ In two of the prose reading tests, compre- ' 
• hension questions were asked at the conclusion of the\ral reading. Although 
the data have not been completely analyzed as of thiS' time, . it is' apparent 
that th^se Aiildren unable to perform the analytic and synthetic tasks, within 
the test-schemes were also not able to read the connected prose. Apparently 
the elemental processes we were seeking to measure did provide the basis of 
, beginning 'reading achievement. * / * - / 

The low correlation between scores on the subksts and the bipiodality 
of performances on subtests indicate that we may haX approached the ele- 
mental level 6f processes within oiir test-schemes.. Doing well on one sub- ' 
test did .^of predict success as additional sources oj^dif ficulty were added. . 
We were able to arrive at and identify the |)recise source of difficulty 
beyond -^hich the child could not succeed^ This manner of testing identifies 
specific levels of mastery or ignornace for individual students, and has " 
the power to identify areas. in which a beginning reading program needs 
modification in order for some children to succeed. Likewise, tho^e" stu- 
dents already mastering -elemental- processes may be identified and -moved on 
to more- appropriate learning situations. This method of assessment offers 
promise of factoring out the significant dimensions of beginning reading be- 
haviors and of offering the teacher precise diagnostic information about the 
instructional needs of beginning readers. 



\ 
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